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Aim This paper aims to identify health research priorities that could 
improve the rate of progress in reducing global neonatal mortality 
from preterm birth and low birth weight (PB/LBW), as set out in the 
UN's Millennium Development Goal 4. 

Methods We applied the Child Health and Nutrition Research Ini- 
tiative (CHNRI) methodology for setting priorities in health research 
investments. In the process coordinated by the World Health Orga- 
nization in 2007—2008, 21 researchers with interest in child, mater- 
nal and newborn health suggested 82 research ideas that spanned 
across the broad spectrum of epidemiological research, health poli- 
cy and systems research, improvement of existing interventions and 
development of new interventions. The 82 research questions were 
then assessed for answerability, effectiveness, deliverability maxi- 
mum potential for mortality reduction and the effect on equity us- 
ing the CHNRI method. 

Results The top 10 identified research priorities were dominated by 
health systems and policy research questions (eg, identification of 
LBW infants born at home within 24—48 hours of birth for addi- 
tional care; approaches to improve quality of care of LBW infants in 
health facilities; identification of barriers to optimal home care prac- 
tices including care seeking; and approaches to increase the use of 
antenatal corticosteriods in preterm labor and to improve access to 
hospital care for LBW infants). These were followed by priorities for 
improvement of the existing interventions (eg, early initiation of 
breastfeeding, including feeding mode and techniques for those un- 
able to suckle directly from the breast; improved cord care, such as 
chlorhexidine application; and alternative methods to Kangaroo 
Mother Care (KMC) to keep LBW infants warm in community set- 
tings). The highest-ranked epidemiological question suggested im- 
proving criteria for identifying LBW infants who need to be cared 
for in a hospital. Among the new interventions, the greatest support 
was shown for the development of new simple and effective inter- 
ventions for providing thermal care to LBW infants, if KMC is not 
acceptable to the mother. 

Conclusion The context for this exercise was set within the MDG4, 
requiring an urgent and rapid progress in mortality reduction from 
low birth weight, rather than identifying long-term strategic solu- 
tions of the greatest potential. In a short-term context, the health 
policy and systems research to improve access and coverage by the 
existing interventions, coupled with further research to improve ef- 
fectiveness, deliverability and acceptance of existing interventions, 
and epidemiological research to address the key gaps in knowledge, 
were all highlighted as research priorities. 
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The UN's Millennium Development Goal 4 (MDG4) states 
that childhood mortality should be reduced by two thirds 
between 1990 and 2015, but assessments show that the 
progress in mortality reduction has been disappointing in 
some countries [1,2]. The main reason usually proposed to 
explain slow progress is insufficient knowledge on how to 
implement existing cost-effective interventions and achieve 
greater coverage of these interventions in low-resource set- 
tings [3] . Generating this knowledge is a task for health re- 
search that should aim to improve efficiency, effectiveness 
and equity in implementation of child survival interven- 
tions in low and middle-income countries. The most recent 
World Health Report published by the World Health Orga- 
nization (WHO) in 2012, entided "No Health without Re- 
search", has also focused on this issue [4,5] . Through this 
flagship report, the WHO tried to highlight the importance 
of health research in reducing the burden of disease and 
disability in the world and "...to provide new ideas, innova- 
tive thinking, and pragmatic advice for member states on how 
to strengthen their own health research systems" [5] . 

To assist policy makers and donors alike in understanding 
the potential of different research avenues to contribute to 
reducing the burden of disease and disability, the Child 
Health and Nutrition Research Initiative (CHNRI) recently 
developed a methodology that allows systematic listing and 
transparent scoring of many competing research options, 
thus exposing their strengths and weaknesses [6-8] . The 
Department of Maternal, Newborn, Child and Adolescent 
Health and Development (MNCAHD) of the WHO has 
used this methodology to identify health research priorities 
to tackle five major causes of child deaths, which are 
thought to underlie two-thirds of all child deaths globally 
[9]. The most recent estimate reported 8.8 million deaths 
in children younger than 5 years worldwide in the year 
2008, and the main causes were pneumonia (18%), diar- 
rhea (15%), preterm birth complications (12%), neonatal 
infections (10%) and birth asphyxia (9%) [9]. The results 
of the CHNRI process coordinated by the World Health 
Organization to identify research priorities to reduce the 
mortality burden from childhood pneumonia, diarrhea, 
birth asphyxia and neonatal infections have already been 
published ]10-13]. 

The cause "preterm birth complications", which comprises 
the old causes "preterm birth" and "low birth weight" (PB/ 
LBW), is on a continuous rise as a proportional cause of 
child deaths globally and it may become the leading cause 
over the next decade, as the importance of infectious dis- 
eases steadily decreases. Currently, PB/LBW are estimated 
to cause around 1 million deaths each year [9] . Unfortu- 
nately, research interest and investments in preventing neo- 
natal deaths from PB/LBW have not been commensurate 
with the importance of LBW as the leading child killer 
[ 14, 1 5] . In this paper, we present the results of the CHNRI 



process to set research priorities to reduce the mortality 
burden from PBA-BW within a context and time frame of 
the UN's Millennium Development Goal 4. 

METHODS 

The CHNRI methodology for setting priorities in health 
research investments was proposed as a tool that could be 
used by those who develop research policy and/or invest 
in health research [6-8]. This aims to assist policy makers 
to understand the full spectrum of research investment op- 
tions and the potential risks and benefits that can result 
from investments in different research. It also assesses the 
likelihood of achieving reductions of persisting burden of 
disease and disability through investments in health re- 
search. The CHNRI methodology has 4 stages: (i) input 
from investors/policy-makers (who define the context and 
the criteria for priority setting); (ii) input from a larger 
group of technical experts (who propose, list systematical- 
ly and then independently score many research ideas); (iii) 
input from other stakeholders (who agree differential 
weights for the chosen priority-setting criteria according to 
wider societal system of values) ]6-8;16]; and (iv) compu- 
tation and discussion of the scores and analysis of the 
agreement between experts. The conceptual framework for 
the CHNRI methodology is shown in Figure 1 and Table 
1 . More detailed explanation has been published elsewhere 
[6-8; 16] and is also available in the Online Supplementary 
Document (table wl). 

Input from investors/policy makers 

The WHO Maternal, Newborn, Child and Adolescent 
Health and Development program (MNCAHD) coordinat- 
ed a large international exercise in 2007-2008, involving 
more than 200 experts from about 80 different countries, 
to identify health research priorities that could directly 
tackle the main causes of global child mortality: pneumo- 
nia, diarrhea, birth asphyxia, preterm birth/low birth 
weight and neonatal infections. The aim was to inform key 
global donors, public investors in health research, and in- 
ternational agencies on research investment policies that 
could support efforts to accelerate the progress toward the 
MDG4. Thus, the context for this exercise was a short-term 
one, set within the MDG4 and requiring an urgent and 
rapid progress in mortality reduction from childhood 
pneumonia rather than identifying long-term strategic so- 
lutions of the greatest potential. While defining this con- 
text, the WHO also recognized the importance of context- 
specific issues at local or regional levels, the large problem 
of pneumonia morbidity, and the beneficial effects of in- 
vestments in the improvement of malnutrition and other 
cross-cutting and cross-sectoral issues ]17,18] . Further de- 
tails are provided in the Online Supplementary Document 
(table wl). 
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PRIORITY SETTING IN HEALTH RESEARCH INVESTMENTS TO ACHIEVE UN's 
MILLENNIUM DEVELOPMENT GOAL 4 - REDUCING CHILDHOOD MORTALITY BY TWO THIRDS BY 2015 



PRIORITY-SETTING CRITERL\: 
CURRENT PRACTICE: 



INTERESTS OF GROUPS, 
LEVEL OF ADVOCACY, 

MEDIA EXPOSURE, 
INTERESTS OF DONORS, 
INDIVIDUAL BIASES, 
DELPHI STUDIES 



ANSWERABILITY IN AN 
ETHICAL WAY 

ATRACTIVENESS OF 
RESEARCH RESULTS, 
NOVELTY, POTENTIAL 
FOR PUBLICATION IN 
HIGH-IMPACT JOURNALS - 



DECISION-MAKING ON INVESTMENTS 
INTO COMPETING 
HEALTH RESEARCH OPTIONS 



PROPOSED HEALTH RESEARCH OPTIONS 
TO GENERATE 
NEW KNOWLEDGE 



PRIORITY-SETTING CRITERIA: 
CHNRI PRACTICE: 



OVERALL IMPACT OF 
PRIORITY-SETTING 
PROCESS ON EQUITY 



GENERATED 

NEW 
KNOWLEDGE 



IMPLEMENTATION: IMPROVED EFFICIENCY OF 

EXISTING HEALTH SYSTEMS, EXISTING 
INTERVEN-TIONS OR DEVELOPMENT OF NEW 
INTERVENTION 



RESULTING REDUCTION 
OF DISEASE BURDEN 
IN THE POPULATION 



ANSWERABILITY IN AN 
ETHICAL WAY 



LIKELIHOOD OF 
EFFICACY AND 
EFFECTIVENESS OF 
INTERVENTIONS AFFECTED 
BY NEW KNOWLEDGE 



LIKELIHOOD OF 
DELIVERABILITY, 
AFFORDABILITY AND 
SUSTAINABILITY 



MAXIMUM POTENTIAL FOR 
REDUCTION OF EXISTING 
DISEASE BURDEN 



Figure 1 CHNRI's conceptual 
framework showing key steps 
required to get from investments in 
ftealth research options to decrease 
in burden of death, disease or 
disability The framework identifies 
criteria to discriminate between 
likelihoods of success of competing 
research options: (i) answerability; 
(ii) effectiveness; (iii) deliverability; 
(iv) maximum potential for disease 
burden reduction; and (v) predicted 
impact on equity in the population 
(right side). These criteria are not 
necessarily what drives investment 
decisions in health research today 
(leftside) [6-8]. 



Table 1 CHNRI's starting framework from which listing of many research options (level of 3-to-5-year research program) and research 
questions (level of individual research papers) were being proposed by technical experts to systematically organize 82 research ideas 



1 Research instrument 


Research avenue 


Epidemiological research 


Measuring the burden 


Understanding risk factors 




Evaluating the existing interventions 


Health policy and 


Studying capacity to reduce exposure to proven health risks 


systems research 


Studying capacity to deliver efficacious interventions 


Research to improve 


Research to improve deliverability 


existing interventions 


Research to improve affordability 


Research to nuprove sustainability 


Research for develop- 


Basic research 


ment of new interven- 


Clinical research 


tions 


Public health research 



Research option 



Research ouestion 



Technical experts were invited to 
use categorization of research av- 
enues and instruments to sys- 
tematically propose a number of 
'research options' within each of 
the avenues; 'research options' 
correspond to the level of 3-to- 
5-y research program 



Technical experts were in-vited to 
propose a number of very specific 
'research questions', correspond- 
ing to the title of indi'vidual re- 
search papers, within each of the 
'research avenues; eventually, after 
consolidation and removing of 
duplicate ideas, 82 such ques- 
tions were retained for scoring 



Input from technical experts 

Individuals -with a wide range of technical expertise and 
regional representation -were recruited to participate. A 
large list of research questions -was drafted by the technical 
expert group based on recent systematic reviews and a sur- 
vey of experts. Eventually, 21 researchers 'with interest in 
child, maternal and newborn health suggested 82 research 
ideas that spanned across the broad spectrum of epidemi- 
ological research, health policy and systems research, im- 
provement of existing interventions and development of 
new interventions. They were organized using the CHNRI 
framework for listing research questions, shown in Table 
1. The expert group then reviewed the questions, refining 
and reformulating them to allow the scoring. The final 
questions were sent to each technical group member for 
scoring. The priority-setting criteria that were adopted 
were: (i) answerability (in an ethical way); (ii) likelihood of 
effectiveness; (iii) likelihood of deliverability, affordability, 
and sustainability; (iv) maximum potential impact on mor- 
tality reduction; and (v) predicted impact on equity. The 



CHNRI framework for scoring research questions is sho'wn 
in Table 2 [7,8] . Further details are provided in the Online 
Supplementary Document (table wl). 

Solicited input from other societal 
stakeholders 

The five criteria for scoring (answerability, efficacy and ef- 
fectiveness, deliverability, disease burden reduction and ef- 
fect on equity) may be perceived to be of varying impor- 
tance and the value given to each criterion may vary 'with 
the perspective of stakeholders. For example, parents who 
have experienced a pneumonia associated death may rate 
mortality reduction much higher than a research funder 
who may value answerability, or a health system planner 
who may be most concerned with deliverability. Hence, 
CHNRI undertook an exercise to poll a 'wide range of stake- 
holders and to weight the criteria based on values assigned 
by these stakeholders, as described elsewhere [16]. The 
weights applied in this exercise are explained in detail in 
the Online Supplementary Document (table wl). 
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Table 2 Questions answered by technical experts to assign intermediate scores for each criterion to 82 competing research ideas* 

CRITERION I: Likelihood that research would lead to new knowledge (enabling a development / planning of an intervention) in an 
ethical way. 

1. Would you say the research question is well framed and endpoints are well defined? 

2. Based on: (i) the level of existing research capacity in proposed research; and (ii) the size of the gap from current level of knowledge to the 
proposed endpoints; would you say that a study can be designed to answer the research question and to reach the proposed endpoints of the 
research? 

3. Do you think that a study needed to answer the proposed research question would obtain ethical approval without major concerns? 
CRITERION 2; Assessment of likelihood that the intervention resulting from proposed research would be effective. 

1. Based on the best existing evidence and knowledge, would the intervention which would be developed / improved through proposed research 
be efficacious? 

2. Based on the best existing evidence and knowledge, would the intervention which would be developed / improved through proposed research 
be effective? 

3. If the answer to either of the previous two questions is positive, would you say that the evidence upon which these opinions are based is of 
high quality? 

^ CRITERION 3: Assessment of deliverability, affordability and sustainability of the intervention resulting from proposed research. 

1 . Taking into account the level of difficulty with intervention delivery from the perspective of the intervention itself (eg, design, standardization, 
safety), the infrastructure required (eg, human resources, health facilities, communication and transport infrastructure) and users of the inter- 
vention (eg, need for change of attitudes or beliefs, supervision, existing demand), would you say that the endpoints of the research would be 
deliverable within the context of interest? 

2. Taking into account the resources available to implement the intervention, would you say that the endpoints of the research would be afford- 
able within the context of interest? 

3. Taking into account government capacity and partnership requirements (eg, adequacy of government regulation, monitoring and enforcement; 
governmental intersectoral coordination, partnership with civil society and external donor agencies; favorable political climate to achieve high 
coverage), would you say that the endpoints of the research would be sustainable within the context of interest? 

CRITERION 4: Assessment of maximum potential of disease burden reduction. 

As this dimension is considered 'independent" of the others, in order to score competing options fairly, their maximum potential to reduce dis- 
ease burden should be assessed as potential impact fraction under an ideal scenario, ie, when the exposure to targeted disease risk is decreased 
to 0% or coverage of proposed intervention is increased to 100% (regardless of how realistic that scenario is at the moment — that aspect will be 
captured by other dimensions of priority setting process, such as deliverability, affordability and sustainability) 
Non-existing interventionst 

Maximum potential to reduce disease burden should be computed as "potential impact fraction" for each proposed research avenue, using the 
equation PIF = [S„_j R (RR,-1)] / [S„.j,„„, F (I?R,-I) ■+ 1] 

where PIF is "potential impact fraction" to reduce disease burden through reducing risk exposure in the population from the present level to 0% 
or increasing coverage by an existing or new intervention from the present level to 100%; RR is the relative risk given exposure level (less than 
1.0 for interventions, greater than 1.0 for risks), P is the population level of distribution of exposure, and n is the maximum exposure level. 
Existing interventions^ 

Maximum potential to reduce disease burden should be assessed from the results of conducted intervention trials; if no such trials were under- 
taken, then it should be assessed as for non-existing interventions. 
Then, the following questions should be answered: 

1. Taking into account the results of conducted intervention trials**, or for the new interventions the proportion of avertable burden under an ide- 
al scenario*, would you say that the successful reaching of research endpoints would have a capacity to remove 5% of disease burden or more? 

2. To remove 10% of disease burden or more? 

3. To remove 15% of disease burden or more? 

CRITERION 5; Assessment of the impact of proposed health research on equity. 

1 . Does the present distribution of the disease burden affect mainly the underprivileged in the population? 

2. Would you say that either (i) mainly the underprivileged, or (ii) all segments of the society equally, would be the most likely to benefit from 
the results of the proposed research after its implementation? 

3. Would you say that the proposed research has the overall potential to improve equity in disease burden distribution in the long term (eg, 10 y)? 

*Possible answers: Yes= 1; No = 0; Informed but undecided answer: 0.5; Not sufficiently informed: blank 

TInterventions that are in the pipeline, or could be envisaged as a possibility, but have not been licensed for implementation yet 
1= Interventions that have been licensed for implementation, but may or may not have been evaluated and implemented 



Computation of the research priority 
scores and average expert agreement 

Completed worksheets were returned to the group coordi- 
nator. The overall research priority score (RPS) was com- 
puted as the mean of the scores for the five criteria [8] , 
weighted according to the input from the stakeholders 
[16], according to the formula: 

jjpj ^ (Cl X 0.96) + (C2 X 0.86) + (C3 X 0.86) + (C4 X 1.75) + (C5 X 0.91) 

5 

where C designates the scores for relevant criteria. 



Average Expert Agreement (AEA) scores were also com- 
puted for each research question as the average proportion 
of scorers that agreed on the 15 questions asked. This is 
computed for each scored research investment option as: 

AEA ^ X ^ scorers who provided most frequent response 
15 N of all scorers 

where q is a question that experts are being asked to eval- 
uate competing research investment options, ranging from 
1 to 15. For further details regarding the choice of meth- 
ods, agreement statistics and interpretation see the Online 
Supplementary Document (table wl). 



www.jogh.org • doi: 10.7189/jogh.02.010403 



4 



JUNE 2012' VOL. 2 NO. 1 • 010403 



Bahl et al. 



RESULTS 

The scores given to all 82 research questions from individ- 
ual experts are presented in Online Supplementary Docu- 
ment (table w2), while the final list of priorities vvith inter- 
mediate and final priority scores for all research questions 
is presented in Online Supplementary Document (table 
w3). In the main body of the paper, Tables 3 and 4 show 
the top ten, and also the bottom-ranked ten ideas, respec- 
tively from the 82 proposed and evaluated research ques- 
tions. The latter three tables transparently present the like- 
lihood for each research question to comply with each of 
the five chosen priority-setting criteria. Research questions 
from three broad research domains (health systems and 



policy research; research to improve the existing interven- 
tions; and epidemiological research) feature in the top 10 
ranked research questions. The identified research priori- 
ties were dominated by health systems and policy research 
questions (eg, identification of LBW infants bom at home 
within 24-48 hours of birth for additional care; approaches 
to improve quality of care of LBW infants in health facili- 
ties; identification of barriers to optimal home care prac- 
tices including care seeking; and approaches to increase the 
use of antenatal corticosteriods in preterm labor and to im- 
prove access to hospital care for LBW infants). These were 
followed by priorities for improvement of the existing in- 
terventions (eg, early initiation of breastfeeding, including 
feeding mode and techniques for those unable to suckle di- 



Table 3 Top lO researcli questions according to tlieir achieved research priority score (RPS), with average expert agreement (AEA) 



related to each question 



1 Rank Proposed research ouestion 


Res. tvpe 


AnSIVERARLE? 


Eeeective? 


Deuverarle? 


BURDEII REDUCT.? 


EOUITARLE? 


AEA (%) 


RPS (weigh) I 


1 


Identification of low birth weight (LBW) infants within 24-48 h of 
birth for additional care among those bom at home 


HPSR 


94 


89 


89 


71 


89 


82.1 


84.2 


2 


Approaches to improve C[uality of care of LBW infants in health 
facilities 


HPSR 


81 


100 


94 


79 


72 


80.8 


83.9 


3 


Identification of cun'ent behaviors, and barriers and supports for 
optimal home care practices, including care seeking for illness 


HPSR 


86 


78 


86 


74 


97 


77.6 


82.7 


4 


Approaches to increase the use of antenatal corticosteriods in 
preterm labor in resource -poor settings 


HPSR 


81 


91 


100 


71 


81 


81.9 


82.4 


5 


Effective interventions for achieving early initiation of breastfeed- 
ing including feeding mode and techniques for those unable to 
suckle directly from the breast 


RIEI 


86 


100 


97 


67 


72 


79.0 


81.5 


6 


Approaches to improve access to care for the subset of LBW 
infants who need hospital care 


HPSR 


94 


82 


78 


76 


81 


74.8 


81.4 


7 


Improved criteria for identifying LBW infants who need to be 
cared for in a hospital 


EFI 


86 


97 


81 


71 


78 


75.4 


80.8 


8 


Effectiveness of improved cord care (eg, chlorhexidine 
application) 


RIEI 


94 


91 


81 


60 


86 


78.7 


78.8 


9 


Comparison of Kangaroo Mother Care (KMC) and alternative 
methods of keeping the LBW infant warm in community settings 


RIEI 


89 


97 


78 


55 


97 


82.8 


78.6 


10 


Approaches to increase the use of antibiotics for premature 
prolonged rupture of membranes in resource-poor settings 


HPSR 


94 


81 


75 


60 


97 


75.7 


78.2 



EPI — epidemiological research, HPSR — health policy and systems research, RIEI — research to improve existing interventions, RDNI — research to develop new 
interventions 



Table 4 The bottom 10 research questions according to their overall research priority score (RPS), with average expert agreement 
(AEA) related to each question 







Res. Type Answerable? 


Effective? 


Deliverable? 


Burden beduct.? 


Equitable? 


AEA (%) 


RPS(weigh) I 


73 


Contribution of preterm birth and intrauterine growth 
retardation to stunting in childhood (increased risk of 
LBW in next generation of girls subiected to stunting) 


EPl 


86 


39 


22 


14 


81 


71.6 


43.6 


74 


Development of safe and effective pharmacological 
methods of stimulating breastmilk supply 


RDNI 


64 


41 


34 


33 


42 


61.8 


41.5 


75 


Approaches to reduce smoking in fathers of unborn 
chidren during pregnancy 


HPSR 


67 


25 


39 


21 


50 


63.2 


37.8 


76 


Development of interventions for activating endogenous 
surfactant production through gene switching 


RDNI 


47 


54 


6 


36 


39 


62.9 


36.2 


77 


Investigating the relationship between sleeping 
arrangements, infections and SIDS in LBW infants 


EPI 


56 


56 


6 


26 


44 


67.6 


35.8 


78 


Determine the degree to which second-hand smoke 


EPI 


64 


42 


22 


10 


56 


70.3 


34.3 


contributes to LBW among non-smoking women 




Development of methods for harmonising the 


















79 


composition of expressed breastmilk to infant 
requirements without constraining output 


RDNI 


50 


59 


13 


19 


42 


67.1 


33.9 


80 


Development of maternal biochemical indicators 
predicting low birth weight 


EPI 


69 


28 


18 


26 


31 


63.3 


33.5 


81 


Investigating the relationship of the home en\i.ronment 
and neurocognitive development of LBW infants 


EFI 


53 


50 


28 


0 


58 


71.1 


31.9 


82 


Development of interventions for activation of HbA 
synthesis to ameliorate early anemia in preterm babies 


RDNI 


53 


46 


6 


21 


39 


67.2 


31.5 



EPI — epidemiological research; HPSR — health policy and systems research; RIEI — research to improve existing interventions; RDNI — research to de- 
velop new interventions 
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rectly from the breast; improved cord care, such as chlorhex- 
idine application; Kangaroo Mother Care (KMC) and other 
methods to keep LBW infants warm in community set- 
tings). The highest-ranked epidemiological question, 
ranked 7th, suggested improving criteria for identifying 
LBW infants who need to be cared for in a hospital. 

The predominance of research questions from the domain 
of health systems and policy research is not surprising, be- 
cause technical experts were asked to define research pri- 
orities that could lead to notable improvements in reduc- 
tion of PB/LBW mortality by the year 2015. This short time 
frame benefited research questions that proposed to iden- 
tify key obstacles to delivery affordability and sustainabil- 
ity of implementation of existing cost-effective interven- 
tions on a larger scale. The exercise also highlighted the 
value of investments that aimed to improve and optimise 
the use of those interventions (alone or in combination) in 
different contexts, and to develop entirely new approaches 
that could assist delivery or acceptance of the existing cost- 
effective interventions. 

Research questions seeking to develop new interventions 
had only three representatives among the 30 highest- 
ranked questions. This is not surprising given the short 
specified time frame (the year 2015) by when it would be 
difficult to envisage new interventions that could have sub- 
stantial impact, as the CHNRI exercise was conducted in 
2007 and 2008. The three ideas that were still encouraged 
by the experts were: (i) the development of new simple and 
effective interventions for providing thermal care to LBW 
infants, if KMC is not acceptable to the mother — which 
was ranked at the high 12th position on the final list; (ii) 
identifying micronutrients whose supplementation im- 
proves functional outcomes including survival in distinct 
subgroups of preterm and growth retarded infants; and (iii) 
development of new simple and effective interventions that 
prevent infections and improve survival, such as new emol- 
lients for massage (see Online Supplementary Document, 
table w3). 

Among the bottom ranked 10 research ideas, five were 
questions related to epidemiological research, while further 
four proposed the development of entirely new interven- 
tions and one was health policy and systems research ques- 
tion. The reasons for their low score vary substantially: the 
ideas on "gene switching to activate endogenous surfactant 
production" or "harmonising the composition of expressed 
breastmilk" were neither considered answerable nor equi- 
table. The proposals to "reduce smoking in fathers of un- 
born children" or "develop maternal biochemical indicators 
predicting low birth weight" were not considered effective 
in mortality reduction. Interventions that should be devel- 
oped from "studying sleeping arrangements, infections and 
SIDS", or "activation of HbA synthesis to ameliorate early 



anemia" were not considered deliverable in the context of 
low and middle-income countries. In all the cases of 10 re- 
search questions with the lowest research priority score, 
there was a minimal, or entirely non-existent, optimism 
toward their possible impact on reduction of PBA-BW with- 
in the context defined for this exercise. 

The CHNRI methodology achieved very good discrimina- 
tion between the 82 research questions, with the final re- 
search priority scores ranging from 84.2 (the highest- 
ranked research priority) to 31.5 (the lowest-ranked) out 
of the maximum 100. Furthermore, there was also a sub- 
stantial gradient in the level of agreement among the scor- 
ers on the priority of the 82 questions, investigated by cal- 
culating "average expert agreement" (AEA). The AEA scores 
ranged from 0.533 to 0.828 (with the theoretical minimum 
of 0.250 and maximum of 1.000). AEA indicates the pro- 
portion of scorers that gave the same most frequent answer 
to an average question they were asked in relation to a spe- 
cific research investment option. Average expert agreement 
values are also presented for the top and bottom 10 re- 
search questions in Tables 3 and 4. Generally, the ques- 
tions over which the greatest level of overall agreement was 
observed among the experts were those that also achieved 
very high overall research priority scores. The greatest point 
of controversy was the research questions on the role of 
psychosocial and physical stress (such as manual labor) to 
preterm birth and intrauterine growth retardation (Online 
Supplementary Document, table w3). 

DISCUSSION 

Investment in global health research today would benefit 
from consensus regarding the context, appropriate invest- 
ment strategies, and co-ordination to achieve significant 
reduction of the disease burden in the foreseeable future. 
The present exercise was designed to assist investors and 
policy makers in making more informed choices on their 
investments in health research on PB/LBW by making ap- 
parent the risks and potential benefits associated with in- 
vestments in a broad spectrum of health research options. 
The expected "profit" from investments is associated with 
generating new knowledge that can be translated into de- 
velopment of new (or improvement of existing) interven- 
tions, which are effective, deliverable, affordable, and can 
reduce the existing burden of disease and disability in an 
equitable way. The risk is associated with research that is 
not likely to be answerable, or that develops products un- 
likely to be effective, deliverable, affordable, or sustainable 
by those who need them most. Investors' preference for 
high-risk investment in health research is particularly ques- 
tionable when it is occurring in a context that requires ur- 
gent progress, such as PBA-BW mortality. The focus on 
complex challenges of implementation (ie, improving 
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health systems, training health workers including poorly 
educated village health workers, improving drug supply 
and delivery at community level, etc.), highlighted in this 
exercise, was reflected in many research questions being 
ranked near the top of the list of overall priorities. 

The context for this exercise was set within the MDG4, re- 
quiring an urgent and rapid progress in mortality reduction 
from low birth weight, rather than identifying long-term 
strategic solutions of the greatest potential. In a short-term 
context, the health policy and systems research to improve 
access and coverage by the existing interventions, coupled 
with further research to improve effectiveness, deliverabil- 
ity and acceptance of existing interventions, and epidemi- 
ological research to address the key gaps in knowledge, 
were all highlighted as research priorities. 

Although the advantages of the CHNRI methodology rep- 
resent a serious attempt to deal with many issues inherent 
to a highly complex process of research investment prior- 
ity setting, there are still concerns over the validity of the 
CHNRI approach and related biases. One of them is relat- 
ed to the fact that many possible good ideas ("research in- 
vestment options") may not have been included in the ini- 
tial list of research options that was scored by the experts, 
and to the potential bias toward items that get the greatest 
press. Another concern over the CHNRI process is that its 
end product represents a possibly biased opinion of a very 
limited group of involved people. In theory, a chosen group 
of experts can have biased views in comparison to any oth- 
er potential groups of experts. Those limitations are de- 
scribed and discussed in greater detail in the Online Sup- 
plementary Document (table wl). 

The implementation of the CHNRI methodology showed 
that, vnthin the context of MDG4, a better balance should 
be achieved between specific domains of health research. 
Along with continuing strategic long-term investments in 
new interventions, which represent high-risk high-profit 
strategies, the CHNRI process suggested that more atten- 
tion should be given to health policy research, health sys- 
tems research, operations research, and research that ad- 
dresses political, economic, social, cultural, behavioral, and 
infrastructure issues surrounding the problem of child 
mortality from PBA-BW These domains of health research 



are rarely recognized as attractive by investors in health re- 
search because their results are unlikely to grab the news- 
paper headlines, get published in journals with high impact 
factors, or lead to patents and commercial products. Yet, 
they can generate new knowledge that can be very helpful 
in achieving real progress in disease burden reduction. The 
identified priorities are also in good agreement with the re- 
search supported by WHO's MNCAHD Department at 
present. They emphasize the evaluation of existing inter- 
ventions and the development and testing of new delivery 
approaches for existing interventions. They also highlight 
the value of research on preventive measures, with research 
on new interventions being downplayed within the short- 
term context. 

CONCLUSIONS 

The context for this exercise was set within the MDG4, re- 
quiring an urgent and rapid progress in mortality reduction 
from PB/LBVv^ rather than identifying long-term strategic so- 
lutions of the greatest potential. In a short-term context, the 
health policy and systems research to improve access and 
coverage by the existing interventions, coupled with research 
to improve deliverability of existing cost-effective interven- 
tions in low resource contexts, and epidemiological research 
to address the key gaps in knowledge, were all highlighted 
as research priorities. These questions are mainly targeted at 
better understanding the barriers toward implementation, 
effectiveness and optimization of use of available interven- 
tions and programmes. If progress toward reduction of glob- 
al PB/LBW mortality is to be improved by 2015, these are 
the research questions that are most likely to be of greatest 
importance. However, very few donors agencies recognize 
the importance of these domains of health research to read- 
ily invest in those options [14,15,18]. The core group of 
CHNRI experts made several serious attempts to influence 
the key donors and point to this gap and serious imbalance 
in health research investing between "upstream" and "down- 
stream" health research and aims to evaluate the results of 
the CHNRI process conducted by the WHO at the levels of 
research output from academic institutions, changes in do- 
nor investment priorities, and health research policy chang- 
es at the main international organizations. 
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